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INTRODUCTION 

The purpos© of this paper i« to suggest priorities for 
thm federal government's elementary and secondary education data 
collection efforts. My sucigestions reflect not only my own 
ideas, but also things I have learned while participating in 
discussions of the National Research Council Committee on 
Indicators of Precollege Science and Mathematics Education. This 
committee has spent a great deal of time over the last six months 
discussing the quality of available data on U.S. elementary and 
secondary science and mathemsxtics education. I have tried to 
indicate where y ideas differ from those voiced by other members 
of the Committee. 

I have organized the paper in three sections: outputs, 
inputs, and private schools. In each case, I consider what we 
would like to know, what the available data are, and 
recommendations for the federal government's data collection 
efforts. 



I. OUTPUTS 



A. Test scores 

1. National Assessment of Educational Progress 



What is happening to the cognitive skill levels of 
children attending American schools? We know much more about 
this question than we did twenty years ago, primarily because of 
the National Assessment of Educational Progress (NAEP). The NAEP 
data have told us, for example, thats 
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— the reading skills of 9-year-old children improved over the 
1970s, and the gap between the reading skills of black children 
and white children closed somewhat | 

— the averagr. science skills of students in all age groups fell 
between 1970 and 1973; the skill levels of 9- and 13-year-old 
students were stable over the period 1973-77, while the average 
science skills of 17-year-olds fell still further. 

While the NREP tests rpcults have been informative, there 
are important questions concerning exactly what the tests 
measure. In particular, many analysts have argued that the NAEP 
tests do not measure higher order learning skills. Other 
analysts have argued that the tests do not even provide good 
measures of children's basic science literacy. The limitations 
of the NAEP tests and other tests of students' cognitive skills 
are worrisome for four related r^asons< 

First, we simply do not know whether the evidence on 
national trends in skill levels would be different if the tests 
provided better measures of cognitive skills, especially the 
critically important higher order skills. 

Second, the lack of evidence on students' higher order 
skills makes it impossible to differentiate among alternative 
explanations for puzzles posed by the NAEP test results. For 
example, the NAEP results indicate that the reading skills of 9- 
year-olds improved over the 1970s, while the reading skills of 
17-year-olds remained stagnant or fell. One possible explanation 
for this pattern is that the NAEP tests do not measure the true 
skill levels of older children as well as they measure thotie of 
younger children, and that our schools have in fact been as 
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successful in educating older students as they have been in 
educat ing younger students. Another possibi lity is thac the 
emphasis on basic skill acquisition in the early grades has had 
deleterious effects on students* acquisition of higher order 
reading skills. If true, this may have implications for how we 
teach children in the early gradea. Yet anoth^*^ possibility is 
that many of the younger children benefited from participating in 
a well-developed Title I compensatory education program, while 
older children, if they participated in Title I at all, did so in 
Title I*s uncertain, early years. We cannot differentiate among 
these possible enplanations for the test score patterns until we 
have better measures of students* higher order reading skills. 

Third, when test scopes are used to assess the quality of 
educational programs, they tend to influence curriculum. The 
content of the NAEP tests may assume this role in the years to 
come as states contract with NPEP to provide detailed scores that 
can be used in st^itewide assessments of the quality of schooling. 
It would be extremely unfortunate if the lack of emphasis on 
higher order cognitive skills in assessment tests led to a 
reduction of emphasis on these skills in the curriculum. 

P fourth reason for concern about test quality is that 
student test scores are the measure of teaching effectiveness 
used in almost all studies of the characteristics o^ effective 
teachers and the determinants of effective teaching. If the 
tests do not measure well the skills that chidren need to learn 
and that good teachers strive to teach, studies of the 
determinants of teaching effectiveness may give very misleading 
results. 
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Recommendat ions : 

1. Continued funding for NflEP should have a very high priority. 
Current plans to increase the frequency with which math and 
science skills are tested should be retained. 

2. The federal government should support Educational Testing 
Service's efforts to develop better NAEP tests. It is 
worthwhile not only to develop better multiple choice tests, 
but also to develop and utilize subtests that provide for 
open-ended responses to questions. This type of test item has 
greater potential for measuring students* higher order 
cognitive skills. 

3. While it is critical to introduce better tests as soon as 
possible, it is important to retain fenough of the old test 
items to permit comparison of new NAEP test results with the 
results of previous tests. 

2. Scholastic Aptitude Tests (SftT) ^nd ftrnerican College 
Tests (ACT) 

In recent years, comparisons among states of average SAT 
scores and average ACT scores have become increasingly popular. 
For example, they have a prominent place in the "Secretary's Wall 
Chart." It is well known that the average score m a state is 
sensitive to the average family income, the percentage of high 
school seniors in the state who take the test, and the perc6»ntage 
of students who attend private schools (Dynarsky, 1983; Howe^ 
1985). Moreover, in New Hampshire, the average SAT sco^e is 
high, in part, because, included in the calculation are the h:. gh 
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scores of students from other states who attend private high 
schools in New Hampshire. This is probably the case in othe^- 
states as well. As a result of the influences on these nonschool 
variables on average SftT and ACT scores, these average scores are 
relatively poor indicators of the quality of public education 
provided to students in particular states. 
Reeommendationi 

If the federal government must publish average SOT and ACT scores 
by state, publish alongside them an adjusted set of scores that 
takes into account the influences of participation rates, family 
income, and private school attendance. Such adjusted scores 
could be calculated relatively easily using multiple regression 
methods. My guess is that these adjusted scores would have a 
somewhat different pattern from the simole average scores. If 
this is the case, discussion of the reasons for the differences 
would be provocative, and maybe even informative. 

3. International flssoeiatipn foy> F- v yaluation of gducation 
Achievement (IFQ) Cross-national Test Comnarisons 

Comparing, at one point in time, the average math and 
science scores of students in different countries poses a host of 
problems. In particular, differences in the quality of national 
•cnool systems is only one of many reasons why average test 
scores differ among countries. Consequently, I am skeptical 
about the possibilities of drawing reliable inferences about U.S. 
education from international comparisons at a singla p oir>t \r, 
time* Comparisons over time offer much better prosoects, 
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however. In particular, it is possible to examine how the 
achievement of U.S. students, as measured on the lEA tests, 
changes over time, and whether the position of U.S. students 
relative to students in other countries changes over time. 
Recommend dtt ions 1 

1. Continue financial support for the lEA testing program, 
emphasizing the need to use test and sample designs that 
permit comparisons over time. 

2. NCES should play a larger role in the implementation of the 
lEA tests in the United States. 

3. Greater effort shoulci be made to administer the lEO tests on a 
regular schedule so that comparisons over time can be made 
more reliably. 

4. The Council of Chief State School Officers should be involved 
in administering the tests. This would improve local 
cooperation and reduce sampling bias due to nonresponse. 

B. Dropout Rates 

One important measure of the extent to which our schools 
accomplish the ambitious goal of educating all students is the 
percentage of students who graduate from high school. Most 
commonly, data are collected on dropout rates, which conceptually 
provide the same information as graduation rates. However, as 
Cook, Ginsberg, and SiWith (1985) have documented, U.S. data on 
dropout rates (and graduation rates) are of very poor quality. 
The most common calculation method, comparing the number of 
students who graduate in year n with the number of students who 
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entered high school in year n-3, is flawed for a number of 
reasons, the most important of which is the high mobility rate of 
American families. A critical effect of mobility is that it is 
not possible, using the standard calculation method, to 
distinguish a student who has left formal schooling entirely from 
a student who has transferred to a school in another 
jurisdiction. This problem is more severe the smaller the 
jurisdiction because mobility across jurisdictions is more 
prevalent. Consequently, dropout rates for individual schools, 
if calculated by the method described above, are probably less 
accurate than dropout rates for individual states — although even 
state dropout rates are influenced by family migration patter>^«^. 

It would be extremttly valuable to have data series that 
provide comparable data on dropout rates for individual schools, 
school districts, and states. Such data are particularly 
important to have at this date as many states tighten 
requirements for high school graduation, requiring, for example, 
that students complete more math and science courses and pass a 
minimum competency exam. One of the adverse consequences of the 
new regulations may be that dropout rates increase. It is also 
likely that the effect of the new graduation requirements on 
dropout rates will be sensitive to the grade level at which 
minimum competency tests are administered and the extent to which 
systematic remedial help is available to students who fail. 
Reliable data permitting comparisons of dropout rates among 
states, and within states over time, would be valuable in 
determining how tightened requirements affect dropouts and 
whether the effects are sensitive to the details of the programs. 
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Recommend at i ons ; 

1. NCES should work with the Council of Chief Stdte School 
Officers (CCSSO) to develop and implement a uniform 
methodology for calculating dropout i^ates. Given the 
sensitivity of many state departments of education to federal 
pressure, achieving agreement will not be easy. I speculate 
that it is more than historical chance that explains the 
differences in methodologies used by individual states to 
calculate dropout rates. States probably have good reasons 
for choosing a particular methodology. Understanding why 
individual states calculate dropout rates as they do would be 
helpful in negotiating movement toward a uniform methodology. 
Consequently, a first step in improving data on dropouts is to 
systematically listen to the reasons dropout rates are 
calculated as they are in the individual states. The Council 
of Chief State School Officers may be an important vehicle for 
soliciting information on methodologies for calculating 
dropout rates, and for achieving agreement c,i a uniform 
methodology. 

2. NCES should encourage, and if possible, fund studies that 

examine whether dropout rates as calculated by applying a r\mu 
uniform methodology to school, school c^istrict, a d state 
level data are close to dropout rates calculated from 
longitudinal data on individual students, such as that 
provided by HS4B. The reason is that following individual 
students over time is unquestionably the best way to learn 
about the dropout rates of students with particular 
characteristics who participate in particular kinds of 
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educational programs. In a world of no budgetary constraints, 
w would want all calculations of dropout ratss to bm dons 
with individual, longitudinal data. If a nsw, common 
methodology for calculating dropout rates with aggregated data 
is indeed satisfactory, then the estimated rates should be 
similar to those calculated with data from HS&B. 

C. Life Outcomes: Earned Income, Occupation, Probability of 
Employment 



While American education has many goals, no one would 
deny that a central one is to prepare students with the skills 
and attitudes that will help them to earn a good living. How 
well do our schools accomplish this goal? This question has been 
hotly contested over the last 25 years, with advocates of the 
"human capital'* approach documenting the accomplishments, while 
others, for example, Christopher Jencks (1972), documenting the 
failures. One point on which all analysts who have studied the 
"economic returns to education" issue agree is that learning more 
requires better data. The data sets used by Jencks and the human 
capital economists provide only minimal information about the 
kinds of education students received. We need datasets that 
provide detailed information on children's schooling as well as 
information on post -schooling careers and income paths. 

Creation of the National Longitudinal Study of the High 
School Class of 1372 (NLS72) was a major step in creating a 
database that provided good information on students' school 
experiences and longitudinal information on their subsequent 
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labor market experiences. fin important limitation in the NLS72 
database, however, is that students were interviewed for the 
first time durinq their senior year in high school, after most of 
the formal schooling was completed. 

The High School and Beyond project is another important 
step in providing good longi^ idinal information on the schooling 
and subsequent labor market experiences of fimerican students. 
From the HS&B data, we have already learned that the type of 
schooling students receive has a marked impact on their cognitive 
skills. I eKpect that subsequent research will tell us a great 
deal about the impact of particular types of schooling on 
subseqent labor market SMperiences. The longitudinal study 
slated to begin in 1988 (NELS) offers even more promise for 
increasing our understanding of the roles formal education plays 
in affecting life outcomes; NELS will conduct baseline interviews 
when children are still in elementary school (grade 8). 
Recommendat ionsi 

1. Continue to fund additional follow-up surveys of both the 
NLS72 and HS&B cohorts. It is important to collect 
information on members of the NLS72 and HS&B samples as 
individuals age. Many important effects cf different types of 
school ing may not become evident unt il individuals reach their 
mid-thirt ies. 

2. Do not reduce the sample sizes in the NLS72 and HSiB follow-up 
surveys. This is critical because many important questions can 
be addressed with these data only if relatively large 

subsamples with particular characteristics are retained. For 
example, Manski (1985) has recently conducted an imoortant 
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study of the characteristics of membprs of the NLS72 sample 
who bf/came teachers. This could be done only because the 
overall sample sise in the follow-'up surveys was sufficiently 
large to include 510 individuals who becc^me teachers. (See 
Section II for more on the Manski study.) 
3. Provide sufficient funding for the NELS project to permit 
inclusion in the follow-up surveys of all members f the 
baseline sample, and to trace individuals who drop out of 
school or transfer from one school to another. Tracing 
students who transfer would permit an analysis of why students 
charge schools^ an important question that we know little 
about. 



IX. INPUTS 



ft. Teachers 
1. S alaries 

Common sense, as well as the results of research on the 
determinarts of school effectiveness, point to the importance of 
teachers in the education process. The quality of the teachers 
in American schools depends on the career decisions millions of 
college graduates make about which occupation to enter, and how 
long to remain in that occupation. While many factors influence 
the attractiveness of alternative occupations, one critical 
factor is monetary compensation. For this reason it is important 
to collect annual data on the salaries of teachers relative to 
salaries in other occupations. 

The evidence that relative salaries affect the career 
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decisions of teachers is not overwhelming (to a large extent 
because the research is difficult to do), but evidence does 
exist. For example, studies have shown that teachers* mobility 
decisions depend on salaries (eg. Baugh and Stone, 138S). 
Comparison of student test score trends during the 1970s with 
teacher salary trends is also suggestive. Over the period 
1970-1981, students' skill levels in the physical sciences fell 
much more dramatically than skill levels in biology did. During 
the years 1974-81 (the closest years for which I could find the 
relevant data), the pay premium for a college graduate trained in 
biology who took a job in business or industry instead of 
becoming a teacher grew from 12 percent of a beginning teacher's 
salary to 31 percent. The comparable pay premium for a graduate 
^-rained in physics grew from 39 percent to 86 percent (Bacharach 
et al., 1984, p. 66). These patterns suggest that one of the 
reasons for the test score decline in science was the increasing 
difficulty in attracting qualified science teachers, and that 
this problem was more acute for physical science teachers than 
for biology teachers. 

There are numerous difficulties in compiling comparative 
salaries of teachers relative to salaries in other occupations. 
For example, should the 9 or 10 month salaries of teachers be 
inflated to make them comparable with the 11 or 12 month salaries 
in other professions? There is no consensus on the answer to 
this question, and the attractiveness of teaching salaries at any 
one point in tite is sensitive to the decisior. To my mind, 
however, the key value of relative salary data lies in 
comparisons over time. For examole, given that the job of 



352 



teaching is quite different from that of working in an industrial 
biology laboratory, we do not know whether college graduates 
trained in biology will find it more attractive to take a ♦15,000 
job teaching for 10 months, or the 11 month biolDgy job that pays 
12 percent more. It is reasonable to assume, however, that 
schools are less able to attract talented biology teachers when 
the pay premium for working in industry grows to 31 percent. 
RagQfnmendat ioni 

The federal government should publish on an annual basis 
comparisons of salaries in teaching with those in other 
occupations. The comparisons should be presented separately for 
each academic field. Useful comparisons would be starting 
salaries, and salaries for individuals with ten years of work 
experience. Data on starting salaries are collected currently by 
the Placement Center of Northwestern University, and are 
published by the National Educational ftssociation. Conseauent ly, 
it may not be necessary for NCES to do all of the data 
collection. In fact, it may be efficient to contract with 
Northwestern to collect comparable salary data for experienced 
workers. However accomplished, it is important that annual data 
be available to assess trends in the salaries of beginning 
teachers and experienced teachers relative to salaries in other 
occupat ions. 

2, Quality of the T faghing Stock 

Has the decline over the last IS years in teaching 
salaries relative to salaries in other occupations led to a 
reduction in the quality of college graduates choosing to become 
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teachers'' The evidence on the average SflT scores of new teachers 
suggests that this is the case. fis Vance and Schlechty (1982) 
have documented, thesr scores have fallen quite dramatically in 
recent years. 

To my mind, it is important to collect data on the SflT 
scores of college graduates who enter teaching. The reason is 
that this gives an indication of how bright college graduates 
per ceived the attract iveness of teaching relat i ve to that of 
other occupations. I want to point out that several members of 
the Committee on Indicators of Precollege Science and Mathematics 
Education disagree with this suggestion. They argue that there 
is no evidence indicating that a teacher's SflT score is 
correlated with teaching effectiveness. fllso, iiot all college 
graduates who enter teaching have taken the SflT. Consequently, 
it IS not clear what the scores are telling us. 

I believe that the criticisms of my fellow Committee 
members are important. On the othe*^ side, however, I think about 
the hard questions state legislators will ask Chief State School 
Officers in the coming years about wnether the large increases in 
teacher salaries that many legislatures are passing have 
influenced the success of public schools in attracting talented 
college graduates to teaching. To my mind, SflT score 
information, if presented carefully, could help us in answering 
the legislators* question. 
Reccmmendat ions i 

1. NCES should attempt to collect information on the SflT scores 
of college graduates entering teaching. This effort will 
probably require significant collaboration with state teacner 
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3. Teacher mobility patterns and what thev mean 

The chart on page 27 of tn© NCES publication, Indicators 
of Education Status and Trends (1985) , is provocative. It shows 
that the SAT scores of individuals who left teacning after a few 
years are higher on average than the scores of teachers who 
remained in the classroom. How should we interpret this 
information? Mas this pattern different twenty years ago? ftre 
there policy changes that might alter this pattern? Would higher 
salaries enable school districts to retain teachers with high 
academic ability? Me simply do not know. It may be that 
teachers' decisions about how long to remain in the classroom are 
sensitive to salaries. It is also possible that many 
academically talented college graduates plan to teach for a few 
years, and then move or to a new challenge, such as law school, 
and that these plans would not be altered by moderate changes in 
teacher salaries. 



presented on page 27 of Indicators and to inform policy 
discussions about how to attract and retained talented college 
graduates into teaching, we need to learn more about the 
determinants of teachers* career decisions. Me also need this 
information to improve the models that are used to predict 
teacher shortages and surpluses. To my knowledge, in no existing 
model of the teacher labor market (and I include the NCES model 
in this category) is the supply of new entrants or the turnover 
rate viewed as being sensitive to salaries. This is ironic in 
the sense that increases in teacher salaries are a common theme 
in the wave of current school re"^orm legislation aimed at coping 
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descript ive patterns 



such as the one 
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with tho shortages predicted by the demand and supply models. 

Why don't we know more about the determinants of teacncr 
career patterns? One reason is that research on this set of 
issues requires data on the career decisions teachers make over 
time and on the attributes of their options. Little such data 
exist, although there are opportunities to create more at 
reasonable cost. 
Recommendat ions: 

1. Continue follow-up surveys of the NLS72 sample. Manski (1985) 
has identified 510 individuals in that sample who became 

teachers. One of the many benefits of following this sample 
as its members age is that we could study why some teachers 
remained in the classroom and others did not. It would be 
valuable in future follow-up surveys to include questions 
about the reasons for occupational changes, and about salaries 
before and after job changes. 

2. NCES should ask Manski, Schlechty, and other researchers who 
have studied teachers* careers with the NLS7S data about how 
the HS4B and NELS follow-up surveys could be structured to 
overcome limitations of the NLS72 data for studying this set 
of issues. 

3. The Current Population Survey (CPS) has a limited longitudinal 
component that can be used to examine the reasons teachers 
change jobs. Baugh and Stone (1982) have used CPS data to 
show that teachers' decisions about whether to change jobs are 
sensitive to salaries. My sense is that more could be learned 
from CPS data about the determinants of teachers' career 
decisions (as well as the cairsBir decisions of workers in other 




occupations) if more attention to this set of questions was 
given in the design of questionnaire items. I encourage NCES 
to work with the Census Bureau in enoloring the possibility of 
using the CPS surveys to learn more about teachers' career 
paths, and specifically about the reasons for job changes. 



B. Capital Accounts 

NCES collects a significant amount of information on the 
capital sxpsndituros of local school districts. There is no 
question that the quality of the physical plant influences the 
quality of life for teachers and students in local school 
districts and that the cost of the physical plant is a 
significant burden on many school districts. There is a question 
in my mind, however, about the usefulness of the data that NCES 
collects on capital account expenditures. I have never seen a 
study that uses these data in a manner that improves our 
understanding of how U.S. education works. 
Recommendat ion 

NCES should explore whether the data it collects on capital 
account expenditures are used, and if so, whether the uses 
justify the cost of collecting and processing these data. 
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III. PRIVATE SCHOOLS 



What roivs do nonpublic school* pl^y in educating 

American children? In recent years research by James Coleman 
(1982) and others using the HS&B data has increased our knowledge 
of the roles that certain types of nonpublic schools, especially 
Catholic schools, play. We still know very little about other 
nonpublic schools, however. I focus my attention on two types of 
nonpublic schools that may be playing an increasing role in 
American education: private schools as an after-hours complement 
to public schools, and for-profit private schools. 

A. After-Hours Private Schools 

A recent article in the New York Times reported that an 
increasing number of American children are attending private 
schools after regular school hours to supplement the instruction 
they receive in public schools. If this is indeed the case, then 
it introduces a new determinant of the skill distribution of 
American students, and another mechanism through which affluent 
families can provide for their children better education than 
that provided to children from poor families. (In Japan, private 
after-hours schools, called Juku, are an important mechanism 
through which middle class families prepare their children for 
the national exams that determine entrance to public 
universities; see Cummings, 1380 for more on Juku.) Little is 
known about after-hours private schools in the U.S., and it seems 
worthwhile to try to learn more about them. 
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Recommend at i ons ! 

!• In the design of the new longitudinal study of Pmerican 
students (NELS), include a set of questions asking whether 
students do attend after-hours schools, and if so, what the 
schools do and what they cost. 

S. Include a similar set of questions in the October CPS survey. 

B. For-Profit Schools 

The 1977 Census of Service Industries resorted the 

existence of 2237 for-profit elementary and secondary schools in 
the U.S. These schools were very small, paid their teachers low 
salaries, and were disproportionately located in the South. ft 
recent New York Times article reported that the numoer of 
for— profit schools is growing. The article described a number oF 
relatively expensive for-profit scnools servir^g students from 
upper middle class families. This description seems different 
from the very limited description of for-profit private scnoois 
that can be gleaned from the 1977 Census of Service Industries. 

It would be worthwhile to learn more aDout the number of 
for-profit elementary and secondary schools in the U.S., where 
they are located, what tuitions they charge, and whom they serve. 
Once we know uhe answers to these questions, we could explore 
whether for-profit schools operate differently from 
not-for-profit schools. This would be valuable in thinking about 
the design of state regulations of private schools, and the 
design of voucher systems — tooics of increasing interest in many 
states. 
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Recowmendat ions : 

1. Include questions in the next Census of Service Industries 
that wi 1 1 provide informat ion about for-profit schools. 

2. Explore whether it is possible to use IRS data to learn about 
trends in the number of for-profit schools, their locations^ 
and their scale of operation. 

IV. CONCLUDING COMMENT 

I would like to express my support for the process NCE3 
has initiated in attempting to improve the quality of federal 
statistics on elementary and secondary education. Asking a large 
number of individuals from different backgrounds for their ideas 
is essential to irnoroving NCES*s contribution to understanding 
U.S. education. It is also, however, an invitation to criticisms 
from many fronts. I admire the decision of the NOES leadersnio 
to solicit suggestions with the inevitable accompanying 
criticisms and the decision to make all of the suggestions 
public. I look forward to reading the suggestions of other 
commentators. 

I conclude my comments with one final suggestion. As the 

NCES staff wades through the many sets of suggestions, it is 
inevitable that many suggestions cannot be implemented because of 
their cost, because of the politics of education in our federal 
system, and for a variety of other reasons. While most users of 
NCES statistics are acutely aware of their limitations, many are 
not aware of the reasons certain types of data are not collected. 
Producing a document that attempts to explain these reasons could 
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be very informative. It is also possible that a statement of the 
reaons why potentially valuable data are not collected could lead 
to changes in the budgetary political conditions that 
prevent the data collection. 
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